Description

For debugging reason TRT nodes attributes was cleared on the symbol copy returned by GetOptimizedSymbol.

This modification allow to save and load graph with TRT node using the following procedure:

# Save
mx.contrib.tensorrt.set_use_tensorrt(True)
executor = mx.contrib.tensorrt.tensorrt_bind(sym, ...)
opti_sym = mx.contrib.tensorrt.get_optimized_symbol(executor)
opti_sym.save('my_opti_sym')

# Load
mx.contrib.tensorrt.set_use_tensorrt(False)
opti_sym = mx.sym.load('my_opti_sym')

Comments

• Once saved, the model can be used with MXNET_USE_TENSORRT set to False.
• When arg_params / aux_params are integrated to TRT_engine you don't need to reinitialize as they will be directly saved in the TRT node attributes. It's still on the user side to check if the weights are in the TRT_node or not, and initialize them properly on the later case
• You still can't save using JSON as some serialized attributes may contain "{", "}" or "EOF".